Hitherto, a device that detects a moving object using an image or a device that tracks a moving object using an image has been known (See Patent Documents 1 and 2, for example). The device described in Patent Document 1 detects a moving object by dividing an image frame into a plurality of small blocks, defining a minimum value of an integrated value of motion vectors for each small block as a background motion vector, and separating the background by subtracting it from the original image frame. The device described in Patent Document 2 sets the color of a moving object according to type in advance, acquires a content rate and variance of the color of the moving object in a target area of an image and determines that there is the type of the moving object corresponding to the set color if the content rate and variance satisfy a predetermined condition. Also, the device described in Patent Document 2 estimates distribution of the color of the background in advance, detects a different-colored image group from a difference from the background color and specifies the location of the moving object by using this different-colored image group.